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Towards Physics-Based Design 


• Many variables & expensive simulations suggest a gradient-based approach 

• Conventional sensitivity analysis intractable 

• Adjoint method efficiently provides unlimited sensitivities 
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The Adjoint Approach 


• The Navier-Stokes equations are solved forward in 
time to give information about the flow field (lift, drag) 

• For design, we need to know how the flow field is 
sensitive to a change in design parameters (chord, 
sweep) 

• Perturbing each parameter, computing the flow field, 
and differencing the output to obtain sensitivity is 
intractable for multiple variables 


• An auxiliary set of adjoint equations are obtained by 
differentiating the governing equations 

• Adjoint equations are solved backward in time to 
determine which parameters impact a given output 
and how 

• This provides efficient sensitivity analysis for all 
inputs simultaneously 
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The Adjoint Equations 
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On time levels 1 and 2, the time derivatives are assumed to be 
discretized with the BDF1 and BDF2 schemes, respectively. Taking 
into account the dependencies on data at time levels n - 2 and /i - 3, 
the adjoint equations are obtained as follows: 
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The time-dependent adjoint equations are 
considerably more complex than the 
Navier-Stokes equations 

Total FUN3D implementation consists of 
nearly 1 million lines of code 

Tremendous amount of software 
infrastructure required 


Page 1 of 4 of the adjoint equations derived and implemented in: 

Nielsen, E.J., and Diskin, B., “Discrete Adjoint-Based Design for 
Unsteady Turbulent Flows on Dynamic Overset Unstructured 
Grids,” A! A A Journal, Vol. 51, No. 6, June 2013. 
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Long-Term Research Effort 


• FUN3D development has involved more than 25 years of dedicated work 

• Weathered numerous programmatic/organizational restructurings 

• Has attracted participation from numerous researchers 

• High-risk / high-payoff in line with NASA’s mission 

• Robust analysis capability is foundational to physics-based design 

• A sizeable user base has grown up with FUN3D development 

• Support for numerous NASA missions 

• FUN3D has been integrated into validated industry processes 

• The increase in capability is quickly adopted by existing users 

• User base continues to grow 

• Broad application has helped to harden the analysis capability 
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Across NASA Missions 


FUN3D supplies critical physics-based aerodynamics for 
a broad range of applications across all Mission Directorates 



“The FUN 3D team has developed a capability that 
continues to find new and unique applications of 
significant importance to the agency. ” 

- Dave Schuster 
NASA Technical Fellow for Aerosciences 
NASA Engineering and Safety Center 
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Across NASA Missions 



Launch Abort 


Supersonic Retropropulsion 


Space Technology 




Due to increased demand from NASA scientists and engineers, FUN3D 
simulations now account for the single largest block of supercomputing cycles 
at the Agency: 12% of NAS, or approximately 200 million hours per year 
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Across the Aerospace Industry 



First private company to achieve orbit 
and dock with the International Space Station 

Primary aerodynamics tool: FUN3D 








FUN3D used for extensive analysis of Falcon 1 
and Falcon 9 rockets, Dragon spacecraft 
Team consults frequently and provides new 
features and capabilities as requested 


“The FUN3D software suite and development 
team have enabled SpaceX to rapidly design, 
build, and successfully fly a new generation of 
rockets and spacecraft. ” 

- Justin Richeson 
Manager, SpaceX Aerodynamics 
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At the Department of Defense 



AMRDEC at Redstone Arsenal 

• Troop safety: airworthiness qualification 

• Dramatic cost savings: fewer tunnel & flight tests 

• Intense demand for timely results on massive 
computing systems 

• Decade of use in direct support of the US warfighter 



Air Force Research Labs 

• Funded on-site training workshop (20 students) ~ 

y K v ' NAVAIR at Patuxent River 

• Hosted on-site training workshop 

• Hired two recent Georgia Tech PhD grads 
• FUN3D development for theses 
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Other Government Agencies 


Argonne 
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LAWRKNCK LIVERMORE 
NATIONAL LABORATORY 


• Gordon Bell Prize awarded to FUN3D / ANL / LLNL team 
• Most prestigious award in High Performance 
Computing community 


Oak 
Ridge 

National Laboratory 

• Joint collaboration improving fuel 
efficiency for 18-wheeler trucks 

• Covered by print and TV news 

• Simulations performed on Jaguar 
• The world’s most powerful 

supercomputer at the time 



NCSA 

• Reduced wind turbine noise 
while maintaining performance 


BMI 

Corporation 
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Across Academia 



• Over 50 students have interned with the team or conducted thesis work 
using FUN3D: 20 MS and PhD theses generated 

• Graduates with hands-on FUN3D experience are highly sought after 



Georgia Tech has played a 
significant role in modeling 
vehicles with moving parts, 
such as helicopters 



“FUN 3D is a vital national resource... the investment that NASA 
has made has generated at least a hundredfold in return on 
investment. . . ” 

- Dr. Marilyn Smith, Georgia Tech 



North Carolina A&T has used FUN3D to optimize 
biologically-inspired micro-air vehicles for the US Army 
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BMI 

Corporation 

• Used FUN3D on world’s largest supercomputer 
to perform the most advanced truck simulations 
ever attempted 

• Developed add-on kits that improve fuel 
mileage by as much as 11% 

• Spin-off company has sold over 
40,000 units 




“FUN3D is a national asset. ” 

- Mike Henderson, 
BMI Founder 
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Long Term R&D Pays Off 


1985 


• ☆ ☆☆ 

1990 1995 2000 2005 


☆ 

2010 2015 


2014: Efficient Sensitivity Analysis 
for Unsteady Flows and General Kinematics 

• FUN3D provides sensitivities for thousands of design parameters at the cost of a 
single simulation using the adjoint approach 

• Culminating publication was recently awarded the H .J.E. Reid Award 


What Has Made It Possible 

• Extensive research in algorithms and turbulence modeling 

• Collaborative software development and automated testing 

• End-to-end high-performance computing strategies 

• Continued reliance on experimental validation 

What is the Impact 

• Design with thousands of variables and high-end CFD simulations now tractable 

• A number of innovations considered seminal in the field of adjoint methods 

• Numerous organizations around the world cite FUN3D as a model for their own efforts 

• Starting to formally address MDO through collaboration with other disciplines 
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Team-Oriented Development 


Team-oriented strategy is critical 

• Broad expertise necessary for broad success 

• High truck number 

Pair programming boosts truck number, improves code quality 
Weekly scrums 

• Very quick summary of “Did, Do, In-the-Way” 

• Scrum master notes impediments 

• No “Death by PowerPoint” 

• Management may attend but cannot talk 

Automated testing of repository commits 

• Every commit screened for adherence to coding standard 

• Each commit triggers a series of regression tests 

• Successful gauntlet means FUN3D is safe to ship 
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Vf* 


Design with High-End CFD 


Multidisciplinary Optimization Using Ground-Based Sonic Boom Metrics 


Adjoint across three coupled disciplines provides 
sensitivities of ground metrics with respect to aircraft OML 
For the first time, aircraft can now be systematically 
designed based on ground metrics such as perceived 
loudness 

Baseline 


Sensitivity of ground 
noise to OML shape 



Design 




Aero Optimization of UH-60 Black Hawk 

• Time-dependent RANS, overset 
grids with complex kinematics 

• Determine optimal shape, pilot 
collective/cyclics 

• Maximize performance while 
trimming via nonlinear constraints 
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So What Have We Learned? 



Great progress has been made, but much remains to be done 
towards achieving the ultimate goal of ModSim 


CFD 

• Cannot accurately predict turbulent separated flows 

• Human intuition for meshing is often misleading 

• Anisotropic mesh adaptation difficult 

• Primal solver must be bulletproof 

• Need high-order methods 

• Sensitivity analysis for chaotic flows 

High-Performance Computing 

• Limited computing resources and expertise 

Multidisciplinary Design 

• Formulating design problem not always straightforward 

• Need for global optimization techniques 

• Discipline coupling is not straightforward 

• Many disciplinary tools not scalable, lack sensitivities 

• Geometry parameterization tools lacking 
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Summary 


Long-term Investment Is Paying Off Substantially... 

• FUN3D is an award-winning tool with broad impact and recognized as a 
national asset 

• Platform for educating the next generation of scientists and engineers 

• An innovation leader in the field of adjoint methods 


...But Significant Challenges Remain 

• HPC 

• Completely lacking this core competency 

• Readily available hardware is orders of magnitude behind SOA 

• MDO must be a fundamental thread woven into everyone’s thought processes 
and tool development 

• Availability of validation data remains critical to building confidence in ModSim 
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